TARDIS Analysis Tool Documentation 📊¶

This tool is designed to analyze and compare TARDIS regression data across different commits. It provides visualization capabilities using Plotly for:

  1. Direct Spectrum Comparison: Compares the luminosity vs wavelength plots across different commits
  2. Residual Analysis: Shows the fractional differences between spectra from different commits

Imports 📥¶

In [1]:
import os
from git import Repo
from tardis_analysis import (
    process_commits, load_h5_data, SPECTRUM_KEYS, PLOTLY_COLORS,
    plot_all_commits_plotly, plot_residuals_plotly, commits
)
import plotly.graph_objects as go
from plotly.subplots import make_subplots

Configuration Settings ⚙️¶

The configuration dictionary defines essential parameters for the analysis:

  • Repository Paths:

    • tardis_repo: Path to TARDIS repository
    • regression_data_repo: Path to regression data repository

  • Parameters:

    • branch: Target branch for analysis (default: "master")
    • n: Number of commits to analyze (default: 2), will be ignored if commits list is provided
    • target_file: Path to HDF5 file containing spectrum solver test data
    • save_plots: Boolean flag to enable plot saving (default: True)
    • gap: Number of days between each commit
    • info: If True, returns a DataFrame with hash, author, and message; otherwise returns a list of hashes

  • Optional Parameters:

    • commits: List of specific commit hashes (commented out by default)
    • output_dir: Custom output directory (defaults to "comparison_plots" in tardis_repo if None)
In [2]:
# commits.calculate_commits(n=10, gap=0, info=True)
In [3]:
config = {
    "tardis_repo": "/home/riddhi/workspace/tardis-main/tardis",
    "regression_data_repo": "/home/riddhi/workspace/tardis-main/tardis-regression-data",
    "branch": "master",
    "n": 3,
    "target_file": "tardis/spectrum/tests/test_spectrum_solver/test_spectrum_solver/TestSpectrumSolver.h5",
    "output_dir": None,
    "commits": ["300e565e83112528faaa76e970057ffb1b13f743", "2a06fdfb60190bbd9b49ff572d78772607138660", "2d775dcd1c486227532f537fc41066e942000e56"],  # Uncomment for specific commits
    # "commits": commits.calculate_commits(n=10, gap=0, info=False), #Uncomment for n commits with custom gap
    "save_plots": True
}

if config["output_dir"] is None:
    config["output_dir"] = os.path.join(config["tardis_repo"], "comparison_plots")

Process Commits ↻¶

This code processes either specific commits or the latest n commits from the TARDIS repository. For each commit, it runs regression tests, generates reference data, and stores the results in the regression data repository. The function returns the processed commit hashes, corresponding regression commit hashes, original HEAD position, and the target file path.

It handles both specific commit inputs (via commits parameter) and default behavior (latest n commits), ensuring proper repository state management throughout the process.

In [4]:
if config.get("commits"):
    processed_commits, regression_commits, original_head, target_file_path = process_commits(
        config["tardis_repo"], config["regression_data_repo"], 
        config["branch"], config["target_file"], 
        commits_input=config["commits"]
    )
else:
    processed_commits, regression_commits, original_head, target_file_path = process_commits(
        config["tardis_repo"], config["regression_data_repo"], 
        config["branch"], config["target_file"], 
        n=config["n"]
    )

Original HEAD of regression data repo: 99f70e6682b6649982c159c0c0fd1d009bf8331f
Processing commit 1/3: 300e565e83112528faaa76e970057ffb1b13f743
Running pytest command: python -m pytest tardis/spectrum/tests/test_spectrum_solver.py --tardis-regression-data=/home/riddhi/workspace/tardis-main/tardis-regression-data --generate-reference -x
Current directory: /home/riddhi/workspace/tardis-main/tardis
Pytest stdout:
============================= test session starts ==============================
platform linux -- Python 3.12.4, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/riddhi/workspace/tardis-main/tardis
configfile: pyproject.toml
plugins: anyio-4.4.0, doctestplus-1.2.1, cov-5.0.0, html-4.1.1, metadata-3.1.1
collected 4 items

tardis/spectrum/tests/test_spectrum_solver.py ssss                       [100%]

=============================== warnings summary ===============================
tardis/io/configuration/config_validator.py:6
tardis/io/configuration/config_validator.py:6
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/configuration/config_validator.py:6: DeprecationWarning: jsonschema.RefResolver is deprecated as of v4.18.0, in favor of the https://github.com/python-jsonschema/referencing library, which provides more compliant referencing behavior as well as more flexible APIs for customization. A future release will remove RefResolver. Please file a feature request (on referencing) if you are missing an API for the kind of customization you need.
    from jsonschema import Draft4Validator, RefResolver, validators

../../../miniforge3/envs/tardis/lib/python3.12/site-packages/jupyter_client/connect.py:22
  /home/riddhi/miniforge3/envs/tardis/lib/python3.12/site-packages/jupyter_client/connect.py:22: DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
  given by the platformdirs library.  To remove this warning and
  see the appropriate new directories, set the environment variable
  `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
  The use of platformdirs will be the default in `jupyter_core` v6
    from jupyter_core.paths import jupyter_data_dir, jupyter_runtime_dir, secure_write

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/transport/montecarlo/montecarlo_main_loop.py:123: NumbaTypeSafetyWarning: unsafe cast from uint64 to int64. Precision may be lost.
    vpacket_collection = vpacket_collections[i]

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/util.py:289: PerformanceWarning: 
  your performance may suffer as PyTables will pickle object types that it cannot
  map directly to c-types [inferred_type->mixed,key->block0_values] [items->Index([0], dtype='int64')]
  
    data.to_hdf(buf, key=os.path.join(path, key))

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/util.py:276: PerformanceWarning: 
  your performance may suffer as PyTables will pickle object types that it cannot
  map directly to c-types [inferred_type->mixed,key->block0_values] [items->Index([0], dtype='int64')]
  
    pd.DataFrame(value).to_hdf(

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/util.py:292: PerformanceWarning: 
  your performance may suffer as PyTables will pickle object types that it cannot
  map directly to c-types [inferred_type->mixed,key->values] [items->None]
  
    pd.Series(scalars).to_hdf(buf, key=os.path.join(path, "scalars"))

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/spectrum/formal_integral.py:380: UserWarning: The number of interpolate_shells was not specified. The value was set to 80.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================== 4 skipped, 8 warnings in 96.75s (0:01:36) ===================

Pytest stderr:

Processing commit 2/3: 2a06fdfb60190bbd9b49ff572d78772607138660
Running pytest command: python -m pytest tardis/spectrum/tests/test_spectrum_solver.py --tardis-regression-data=/home/riddhi/workspace/tardis-main/tardis-regression-data --generate-reference -x
Current directory: /home/riddhi/workspace/tardis-main/tardis
Pytest stdout:
============================= test session starts ==============================
platform linux -- Python 3.12.4, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/riddhi/workspace/tardis-main/tardis
configfile: pyproject.toml
plugins: anyio-4.4.0, doctestplus-1.2.1, cov-5.0.0, html-4.1.1, metadata-3.1.1
collected 4 items

tardis/spectrum/tests/test_spectrum_solver.py ssss                       [100%]

=============================== warnings summary ===============================
tardis/io/configuration/config_validator.py:6
tardis/io/configuration/config_validator.py:6
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/configuration/config_validator.py:6: DeprecationWarning:
  
  jsonschema.RefResolver is deprecated as of v4.18.0, in favor of the https://github.com/python-jsonschema/referencing library, which provides more compliant referencing behavior as well as more flexible APIs for customization. A future release will remove RefResolver. Please file a feature request (on referencing) if you are missing an API for the kind of customization you need.

../../../miniforge3/envs/tardis/lib/python3.12/site-packages/jupyter_client/connect.py:22
  /home/riddhi/miniforge3/envs/tardis/lib/python3.12/site-packages/jupyter_client/connect.py:22: DeprecationWarning:
  
  Jupyter is migrating its paths to use standard platformdirs
  given by the platformdirs library.  To remove this warning and
  see the appropriate new directories, set the environment variable
  `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
  The use of platformdirs will be the default in `jupyter_core` v6

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/transport/montecarlo/montecarlo_main_loop.py:123: NumbaTypeSafetyWarning:
  
  unsafe cast from uint64 to int64. Precision may be lost.

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/util.py:289: PerformanceWarning:
  
  
  your performance may suffer as PyTables will pickle object types that it cannot
  map directly to c-types [inferred_type->mixed,key->block0_values] [items->Index([0], dtype='int64')]

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/util.py:276: PerformanceWarning:
  
  
  your performance may suffer as PyTables will pickle object types that it cannot
  map directly to c-types [inferred_type->mixed,key->block0_values] [items->Index([0], dtype='int64')]

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/util.py:292: PerformanceWarning:
  
  
  your performance may suffer as PyTables will pickle object types that it cannot
  map directly to c-types [inferred_type->mixed,key->values] [items->None]

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/spectrum/formal_integral.py:398: UserWarning:
  
  The number of interpolate_shells was not specified. The value was set to 80.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================== 4 skipped, 8 warnings in 94.76s (0:01:34) ===================

Pytest stderr:

Processing commit 3/3: 2d775dcd1c486227532f537fc41066e942000e56
Running pytest command: python -m pytest tardis/spectrum/tests/test_spectrum_solver.py --tardis-regression-data=/home/riddhi/workspace/tardis-main/tardis-regression-data --generate-reference -x
Current directory: /home/riddhi/workspace/tardis-main/tardis
Pytest stdout:
============================= test session starts ==============================
platform linux -- Python 3.12.4, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/riddhi/workspace/tardis-main/tardis
configfile: pyproject.toml
plugins: anyio-4.4.0, doctestplus-1.2.1, cov-5.0.0, html-4.1.1, metadata-3.1.1
collected 4 items

tardis/spectrum/tests/test_spectrum_solver.py ssss                       [100%]

=============================== warnings summary ===============================
tardis/io/configuration/config_validator.py:6
tardis/io/configuration/config_validator.py:6
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/configuration/config_validator.py:6: DeprecationWarning:
  
  jsonschema.RefResolver is deprecated as of v4.18.0, in favor of the https://github.com/python-jsonschema/referencing library, which provides more compliant referencing behavior as well as more flexible APIs for customization. A future release will remove RefResolver. Please file a feature request (on referencing) if you are missing an API for the kind of customization you need.

../../../miniforge3/envs/tardis/lib/python3.12/site-packages/jupyter_client/connect.py:22
  /home/riddhi/miniforge3/envs/tardis/lib/python3.12/site-packages/jupyter_client/connect.py:22: DeprecationWarning:
  
  Jupyter is migrating its paths to use standard platformdirs
  given by the platformdirs library.  To remove this warning and
  see the appropriate new directories, set the environment variable
  `JUPYTER_PLATFORM_DIRS=1` and then run `jupyter --paths`.
  The use of platformdirs will be the default in `jupyter_core` v6

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/transport/montecarlo/montecarlo_main_loop.py:123: NumbaTypeSafetyWarning:
  
  unsafe cast from uint64 to int64. Precision may be lost.

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/util.py:289: PerformanceWarning:
  
  
  your performance may suffer as PyTables will pickle object types that it cannot
  map directly to c-types [inferred_type->mixed,key->block0_values] [items->Index([0], dtype='int64')]

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/util.py:276: PerformanceWarning:
  
  
  your performance may suffer as PyTables will pickle object types that it cannot
  map directly to c-types [inferred_type->mixed,key->block0_values] [items->Index([0], dtype='int64')]

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/io/util.py:292: PerformanceWarning:
  
  
  your performance may suffer as PyTables will pickle object types that it cannot
  map directly to c-types [inferred_type->mixed,key->values] [items->None]

tardis/spectrum/tests/test_spectrum_solver.py::TestSpectrumSolver::test_initialization
  /home/riddhi/workspace/tardis-main/tardis/tardis/spectrum/formal_integral.py:398: UserWarning:
  
  The number of interpolate_shells was not specified. The value was set to 80.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================== 4 skipped, 8 warnings in 96.94s (0:01:36) ===================

Pytest stderr:


Processed Tardis Commits:
300e565e83112528faaa76e970057ffb1b13f743
2a06fdfb60190bbd9b49ff572d78772607138660
2d775dcd1c486227532f537fc41066e942000e56

Regression Data Commits:
f116f125e5aabc98225ad23bb292c3a9ed2bfa3b
4f7a1dfb1fff1535e08ed23c534fbc1929384ce1
4aa60ed4cc2cda102703d7b6201532c9f5c65eb8

Load HDF5 Data 📊¶

This code loads spectrum data from the regression repository for each processed commit. It initializes an empty list commit_data, then iterates through each regression commit to load the corresponding HDF5 data. For each commit, it checks out the commit, loads wavelength and luminosity data using load_h5_data(), and stores it in the list. After processing all commits, it resets the repository to its original state and returns to the main branch.

In [5]:
commit_data = []
regression_repo = Repo(config["regression_data_repo"])

for reg_commit in regression_commits:
    regression_repo.git.checkout(reg_commit)
    commit_data.append(load_h5_data(target_file_path, SPECTRUM_KEYS))

regression_repo.git.reset('--hard', original_head)
regression_repo.git.checkout('main')

Inspecting HDF5 file: /home/riddhi/workspace/tardis-main/tardis-regression-data/tardis/spectrum/tests/test_spectrum_solver/test_spectrum_solver/TestSpectrumSolver.h5
Top-level keys: ['simulation']
Checking simulation/spectrum_solver/spectrum_integrated
Loaded data for spectrum_integrated: wavelength shape=(10000,), luminosity shape=(10000,)
Checking simulation/spectrum_solver/spectrum_real_packets
Loaded data for spectrum_real_packets: wavelength shape=(10000,), luminosity shape=(10000,)
Checking simulation/spectrum_solver/spectrum_real_packets_reabsorbed
Loaded data for spectrum_real_packets_reabsorbed: wavelength shape=(10000,), luminosity shape=(10000,)
Checking simulation/spectrum_solver/spectrum_virtual_packets
Loaded data for spectrum_virtual_packets: wavelength shape=(10000,), luminosity shape=(10000,)

Inspecting HDF5 file: /home/riddhi/workspace/tardis-main/tardis-regression-data/tardis/spectrum/tests/test_spectrum_solver/test_spectrum_solver/TestSpectrumSolver.h5
Top-level keys: ['simulation']
Checking simulation/spectrum_solver/spectrum_integrated
Loaded data for spectrum_integrated: wavelength shape=(10000,), luminosity shape=(10000,)
Checking simulation/spectrum_solver/spectrum_real_packets
Loaded data for spectrum_real_packets: wavelength shape=(10000,), luminosity shape=(10000,)
Checking simulation/spectrum_solver/spectrum_real_packets_reabsorbed
Loaded data for spectrum_real_packets_reabsorbed: wavelength shape=(10000,), luminosity shape=(10000,)
Checking simulation/spectrum_solver/spectrum_virtual_packets
Loaded data for spectrum_virtual_packets: wavelength shape=(10000,), luminosity shape=(10000,)

Inspecting HDF5 file: /home/riddhi/workspace/tardis-main/tardis-regression-data/tardis/spectrum/tests/test_spectrum_solver/test_spectrum_solver/TestSpectrumSolver.h5
Top-level keys: ['simulation']
Checking simulation/spectrum_solver/spectrum_integrated
Loaded data for spectrum_integrated: wavelength shape=(10000,), luminosity shape=(10000,)
Checking simulation/spectrum_solver/spectrum_real_packets
Loaded data for spectrum_real_packets: wavelength shape=(10000,), luminosity shape=(10000,)
Checking simulation/spectrum_solver/spectrum_real_packets_reabsorbed
Loaded data for spectrum_real_packets_reabsorbed: wavelength shape=(10000,), luminosity shape=(10000,)
Checking simulation/spectrum_solver/spectrum_virtual_packets
Loaded data for spectrum_virtual_packets: wavelength shape=(10000,), luminosity shape=(10000,)
Out[5]:
'Your branch and \'origin/main\' have diverged,\nand have 155 and 3 different commits each, respectively.\n  (use "git pull" if you want to integrate the remote branch with yours)'

Display Plots 📈¶

This cell creates and saves visualization plots using Plotly. It sets up a 2x2 subplot grid for spectrum analysis and generates both direct comparison and residual plots when save_plots is enabled. The plots are saved in the configured output directory as interactive HTML files.

Spectrum Plot¶

The spectrum plot shows the relationship between wavelength ($\lambda$) and luminosity ($L$) for each commit:

$$L(\lambda) = f(\lambda)$$

where:

  • $L(\lambda)$ is the luminosity at wavelength $\lambda$
  • $f(\lambda)$ represents the spectral energy distribution function

Residual Plot¶

The residual plot shows the fractional difference between spectra from different commits:

$$R(\lambda) = \frac{L_2(\lambda) - L_1(\lambda)}{L_1(\lambda)} \times 100\%$$

where:

  • $R(\lambda)$ is the residual at wavelength $\lambda$
  • $L_1(\lambda)$ is the luminosity from the reference commit
  • $L_2(\lambda)$ is the luminosity from the comparison commit
  • The result is expressed as a percentage difference

The code ensures the output directory exists before saving and handles both types of plots: luminosity vs wavelength comparisons and fractional residuals across commits.

In [6]:
fig = make_subplots(rows=2, cols=2, subplot_titles=[f'Luminosity for {key}' for key in SPECTRUM_KEYS])
fig.update_layout(height=800, showlegend=True)

if config["save_plots"]:
    os.makedirs(config["output_dir"], exist_ok=True)
    plot_all_commits_plotly(commit_data, SPECTRUM_KEYS, config["output_dir"], commit_hashes=processed_commits)
    plot_residuals_plotly(commit_data, SPECTRUM_KEYS, config["output_dir"], commit_hashes=processed_commits)

Saved Plotly plot as /home/riddhi/workspace/tardis-main/tardis/comparison_plots/all_commits_spectrum_comparison.html
Saved Plotly residual plot as /home/riddhi/workspace/tardis-main/tardis/comparison_plots/residuals_all_commits.html
In [7]:
plot_all_commits_plotly(commit_data, SPECTRUM_KEYS, config["output_dir"], commit_hashes=processed_commits)

Saved Plotly plot as /home/riddhi/workspace/tardis-main/tardis/comparison_plots/all_commits_spectrum_comparison.html
In [8]:
plot_residuals_plotly(commit_data, SPECTRUM_KEYS, config["output_dir"], commit_hashes=processed_commits)

Saved Plotly residual plot as /home/riddhi/workspace/tardis-main/tardis/comparison_plots/residuals_all_commits.html
In [ ]: